What values should an agent align with?

The pursuit of values drives human behavior and promotes cooperation. Existing research is focused on general values (e.g., Schwartz) that transcend contexts. However, context-specific values are necessary to (1) understand human decisions, and (2) engineer intelligent agents that can elicit and align with human values. We propose Axies, a hybrid (human and AI) methodology to identify context-specific values. Axies simplifies the abstract task of value identification as a guided value annotation process involving human annotators. Axies exploits the growing availability of value-laden text corpora and Natural Language Processing to assist the annotators in systematically identifying context-specific values. We evaluate Axies in a user study involving 80 human subjects. In our study, six annotators generate value lists for two timely and important contexts: Covid-19 measures and sustainable Energy. We employ two policy experts and 72 crowd workers to evaluate Axies value lists and compare them to a list of general (Schwartz) values. We find that Axies yields values that are (1) more context-specific than general values, (2) more suitable for value annotation than general values, and (3) independent of the people applying the methodology. Supplementary Information The online version contains supplementary material available at 10.1007/s10458-022-09550-0.


Instructions
Thank you for joining the Axies evaluation! Please find the instructions below.

Introduction
We have a list of personal values relevant to a specific context of discussion, based on the input that participants provided to a survey (additional details follow). Here, we are asking you to evaluate clarity and completeness of the value list.

Value list
Values are abstract motivations that justify opinions and actions, linked to personal feelings and desirable goals. In this experiment, values are represented by a name, a defining goal and a set of keywords. Defining goals and keywords are used to describe the value and capture its meaning in the context under discussion. For example, in the context of browsing the internet, the value of 'privacy' can be represented as follows: • Value: privacy • Keywords: anonymity, data collection, third-party cookies • Defining goal: Being able to control the extent to which one's personal information is shared with others.

Survey context (Covid)
In April, Dutch citizens answered a survey about ways for lifting COVID-19-related measures. In this survey, participants were proposed eight different options, and could choose one or more of them (for example, "All restrictions are lifted for immune people" and "Nursing homes allow visitors again"). They were then invited to explain each chosen option with a motivation.

Survey context (Energy)
The South-West Friesland municipality (in the Netherlands) organized a survey in the scope of transition to renewable energy, with the goal of reaching 50% of renewable energy generation by 2030. They offered multiple options to the participants, who could choose one or more of them (such as "The municipality takes the lead" and "The market determines what's coming"). Participants were then invited to explain each chosen option with a motivation.
Here you can see all values present in the value list, each accompanied by defining goal and keywords. Please indicate how clearly you understand the concept described by each value.

Ambiguity evaluation
You are shown some pairs of values, each accompanied by its defining goal and keywords. Please indicate how motivationally distinct (i.e. to which extent can they be distinguished through their defining goals and keywords) you think the two described concepts are, in a 5-points scale. Please keep in mind that values can have overlap, but still be distinct. For instance, the values of 'achievement' (personal success through demonstrating competence according to social standards) and 'power' (social status and prestige, control or dominance over people and resources) are somewhat similar. However, their distinction comes from the fact that 'achievement' emphasizes the active demonstration of successful performance, whereas 'power' emphasizes the attainment or preservation of a dominant position within the social system.

Opinion annotation
You are sequentially shown the motivations (with the choices) that participants gave in the survey.
Please, indicate which value(s) lie behind each motivation (the related survey choice is there just to help you understand the context of the motivation. The value(s) you indicate should be specifically present in the motivation. On the right side of the screen, you can see the full list of values. For each motivation, you have the following options: • There is a value lying behind the motivation. You can do one or both the following actions and then click on "Submit": ○ Select from the list the value(s) lying behind the motivation. ○ The value lying behind the motivation is not in the list. Add it in the slot at the bottom of the list. Optionally, you can explain the reason why you added it, and some words from the motivation that are related to it. • There is no value lying behind the motivation, or the motivation is unclear. Click on "No values" to get the next motivation.